Papers with Instruction-based Continual Learning
InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions (2024.naacl-long)
Copied to clipboard
| Challenge: | In order to perform downstream tasks, Large Language Models (LLMs) need continual adaptation without catastrophic forgetting. |
| Approach: | They propose a new paradigm that allows for continual adaptation without catastrophic forgetting . they propose to replay previous data based on task similarity with instructions . |
| Outcome: | The proposed method improves performance over 16 tasks with different training orders. |